55 research outputs found
Bimodal network architectures for automatic generation of image annotation from text
Medical image analysis practitioners have embraced big data methodologies.
This has created a need for large annotated datasets. The source of big data is
typically large image collections and clinical reports recorded for these
images. In many cases, however, building algorithms aimed at segmentation and
detection of disease requires a training dataset with markings of the areas of
interest on the image that match with the described anomalies. This process of
annotation is expensive and needs the involvement of clinicians. In this work
we propose two separate deep neural network architectures for automatic marking
of a region of interest (ROI) on the image best representing a finding
location, given a textual report or a set of keywords. One architecture
consists of LSTM and CNN components and is trained end to end with images,
matching text, and markings of ROIs for those images. The output layer
estimates the coordinates of the vertices of a polygonal region. The second
architecture uses a network pre-trained on a large dataset of the same image
types for learning feature representations of the findings of interest. We show
that for a variety of findings from chest X-ray images, both proposed
architectures learn to estimate the ROI, as validated by clinical annotations.
There is a clear advantage obtained from the architecture with pre-trained
imaging network. The centroids of the ROIs marked by this network were on
average at a distance equivalent to 5.1% of the image width from the centroids
of the ground truth ROIs.Comment: Accepted to MICCAI 2018, LNCS 1107
FNOSeg3D: Resolution-Robust 3D Image Segmentation with Fourier Neural Operator
Due to the computational complexity of 3D medical image segmentation,
training with downsampled images is a common remedy for out-of-memory errors in
deep learning. Nevertheless, as standard spatial convolution is sensitive to
variations in image resolution, the accuracy of a convolutional neural network
trained with downsampled images can be suboptimal when applied on the original
resolution. To address this limitation, we introduce FNOSeg3D, a 3D
segmentation model robust to training image resolution based on the Fourier
neural operator (FNO). The FNO is a deep learning framework for learning
mappings between functions in partial differential equations, which has the
appealing properties of zero-shot super-resolution and global receptive field.
We improve the FNO by reducing its parameter requirement and enhancing its
learning capability through residual connections and deep supervision, and
these result in our FNOSeg3D model which is parameter efficient and resolution
robust. When tested on the BraTS'19 dataset, it achieved superior robustness to
training image resolution than other tested models with less than 1% of their
model parameters.Comment: This paper was accepted by the IEEE International Symposium on
Biomedical Imaging (ISBI) 202
Receptivity of an AI Cognitive Assistant by the Radiology Community: A Report on Data Collected at RSNA
Due to advances in machine learning and artificial intelligence (AI), a new
role is emerging for machines as intelligent assistants to radiologists in
their clinical workflows. But what systematic clinical thought processes are
these machines using? Are they similar enough to those of radiologists to be
trusted as assistants? A live demonstration of such a technology was conducted
at the 2016 Scientific Assembly and Annual Meeting of the Radiological Society
of North America (RSNA). The demonstration was presented in the form of a
question-answering system that took a radiology multiple choice question and a
medical image as inputs. The AI system then demonstrated a cognitive workflow,
involving text analysis, image analysis, and reasoning, to process the question
and generate the most probable answer. A post demonstration survey was made
available to the participants who experienced the demo and tested the question
answering system. Of the reported 54,037 meeting registrants, 2,927 visited the
demonstration booth, 1,991 experienced the demo, and 1,025 completed a
post-demonstration survey. In this paper, the methodology of the survey is
shown and a summary of its results are presented. The results of the survey
show a very high level of receptiveness to cognitive computing technology and
artificial intelligence among radiologists
- …